Event Based Video Indexing by Intermodal Collaboration

In this paper, we propose event based video indexing, which is a kind of indexing by semantical contents. To achieve this, we exploit the idea of intermodal collaboration, i.e. collaborative processing taking account of the semantical dependency between multimodal information streams consisting of visual, auditory, and text (closed caption: CC) streams. The proposed method attempts to make temporal correspondence between visual and CC streams and to index shots in the visual stream by detecting the events. From experimental results for broadcasted sports video of American football games, we obtained the recall rate of 86% and the precision rate of 74% for indexing accuracy. The results indicate that the method is e ective for video indexing.